.. _Tutorial: using NeurEco Python API for a Tabular Regression problem:

Tutorial: using NeurEco Python API for a Tabular Regression problem
=======================================================================

The following section uses the test case :std:ref:`Metamaterial Antennas test case`. This test case is included in the NeurEco installation package.

Build a model
---------------

* Create an empty directory (MetamaterialAntenna Example), extract the :std:ref:`Metamaterial Antennas test case` test case data there. The created directory contains the following files:

  * x_test.csv
  * y_test.csv
  * x_train.csv
  * y_train.csv

* Import the required libraries (NeurEco and NumPy):

.. code-block:: python

  from NeurEco import NeurEcoTabular as Tabular
  import numpy as np

* Load the training data:

.. code-block:: python

  x_train = np.genfromtxt("x_train.csv", delimiter=";", skip_header=True)
  y_train = np.genfromtxt("y_train.csv", delimiter=";", skip_header=True)

* Initialize a NeurEco object to handle the **Regression** problem:

.. code-block:: python

  builder = Tabular.Regressor()

All the methods provided by the **Regressor** class, can be viewed by calling the *__method__* attributes:

.. code-block:: python

  print(builder.__methods__)

.. code-block:: text
    
  *** NeurEco Tabular Regressor methods: ***
  - load
  - save
  - delete
  - evaluate
  - build
  - get_input_count
  - get_output_count
  - load_model_from_checkpoint
  - get_number_of_networks_from_checkpoint
  - get_weights
  - export_fmu
  - export_c
  - export_onnx
  - export_vba
  - compute_error
  - plot_network
  - forward_derivative
  - gradient
  - set_weights
  - perform_input_sweep

To understand what each parameter of any method does and how to use it, print the doc of the method:

.. code-block:: python

  print(builder.export_c.__doc__)

.. code-block:: text

    exports a NeurEco tabular model to a header file
    :param h_file_path: path where the .h file will be saved
    :param precision: string: optional: "float" or "double": precision of the weights in the h file
    :return: export_status: int: 0 if export is ok, other if otherwise.

* To build the model, run the **build** method with the building parameters adjusted to the problem at hand (see :std:ref:`Build NeurEco Regression model with the Python API`). For this example, the outputs are normalized per feature (meaning that each output is normalized apart, see :std:ref:`Normalizing the data`):

.. code-block:: python

  builder.build(input_data=x_train, output_data=y_train,
            # the rest of these parameters are optional
            write_model_to="./MetamaterialAntennas/MetamaterialAntennas",
            checkpoint_address="./MetamaterialAntennas/MetamaterialAntennas.checkpoint",
            outputs_normalize_per_feature=True)

* When **build** is called, NeurEco starts the building process:

.. code-block:: text

  Validation Percentage will be used to get the validation data. This is due to:
  - one or all the validation data is set to None
  - validation indices is set to None

   info >
   info >      _   __                ______
   info >     / | / /__  __  _______/ ____/________
   info >    /  |/ / _ \/ / / / ___/ __/ / ___/ __ \
   info >   / /|  /  __/ /_/ / /  / /___/ /__/ /_/ /
   info >  /_/ |_/\___/\__,_/_/  /_____/\___/\____/
   info >                  === A D A G O S ===
   info >
   info > Version: 4.01.2474.0 Compiled with MSVC v1928  Oct 12 2022 Matlab runtime:no
   info > OpenMP: yes
   info > MKL: yes
   info > Reading data files...
   info > Reading Data from C:/Users/Sadok/AppData/Local/Temp/tmp3wmwwjv4/inputs_tab_train.npy
   info > Reading Data from C:/Users/Sadok/AppData/Local/Temp/tmp3wmwwjv4/outputs_tab_train.npy
   info > build for: 6 outputs and 7 inputs and 457 samples.
   info > Building Model

During the build NeurEco saves the intermediate modes to the checkpoint file (defined by the parameter **checkpoint_address**). 
To load and use the intermediate models from this checkpoint: 

* Create a new NeurEco object in which to load the model:

.. code-block:: python

  model = Tabular.Regressor()
 
* Determine how many intermediate models the checkpoint contains:

.. code-block:: python

  n = model.get_number_of_networks_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint")

* Load any intermediate model from the checkpoint using its id (count starts with zero). For this example, at the moment of running the command :math:`n=6` and the following command loads the intermediate model :math:`n°3 \ (id=2)`:

.. code-block:: python

  model.load_model_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint", 2)

Now **model** is a valid **Regression** model, and can be used as usual. 

* Check the number of trainable parameters each of the intermediate models has:

.. code-block:: python

  for i in range(n):
      print("Loading model", i, " from checkpoint file:")
      model.load_model_from_checkpoint("./MetamaterialAntennas/MetamaterialAntennas.checkpoint", i)
      print("number of trainable parameters in intermediate model --", i, " is:", model.get_weights().size)

.. code-block:: text

  Loading model 0  from checkpoint file:
  number of trainable parameters in intermediate model -- 0  is: 34
  Loading model 1  from checkpoint file:
  number of trainable parameters in intermediate model -- 1  is: 62
  Loading model 2  from checkpoint file:
  number of trainable parameters in intermediate model -- 2  is: 90
  Loading model 3  from checkpoint file:
  number of trainable parameters in intermediate model -- 3  is: 132


Evaluate a model
------------------

* Load the testing data from the CSV files:

.. code-block:: python

  x_test = np.genfromtxt("x_test.csv", delimiter=";", skip_header=True)
  y_test = np.genfromtxt("y_test.csv", delimiter=";", skip_header=True)

* Create a **Regressor** object to use for the evaluation:

.. code-block:: python

  evaluator = Tabular.Regressor()

.. note::
    It is possible to use the already existing **Regressor** object **builder** when the evaluation is done just after the **build**, and **builder** is still available.

* Load the built model:

.. code-block:: python

  load_state = evaluator.load("./MetamaterialAntennas/MetamaterialAntennas")

.. note::
    When building or evaluating a NeurEco model, all the used paths do not necessarily need to have an extension when they are passed as parameters to a NeurEco method.

* To extract information from the loaded model, such as the number of inputs, the number of outputs and the weights array, run:

.. code-block:: python

  n_inputs = evaluator.get_input_count()
  n_outputs = evaluator.get_output_count()
  weights = evaluator.get_weights()
  print("Number of Inputs:", n_inputs)
  print("Number of Outputs:", n_outputs)
  print("Number of trainable parameters:", weights.size)


.. code-block:: text

  Number of Inputs: 7
  Number of Outputs: 6
  Number of trainable parameters: 458

* To plot the network graph (this operation requires *matplotlib* library installed, see :std:ref:`Plot a NeurEco network Regression Python API`):

.. code-block:: python

  evaluator.plot_network()

.. figure:: ./images/MetamaterialAntennaPythonNetworkPlot.png
    :width: 800
    :alt: MetamaterialAntennaPythonNetworkPlot
    :align: center

    Python API operations: plotting a network: test case - MetamaterialAntennas 


* To evaluate the model on the test data:

.. code-block:: python

  neureco_outputs = evaluator.evaluate(x_test)
  l2_error = evaluator.compute_error(neureco_outputs, y_test)
  print("L2 relative error (%):", 100 * l2_error)

.. code-block:: text

  L2 relative error (%): 1.4545507588248332

.. note::
  During evaluation, the normalization is carried out by the model and its parameters are not relative to the data set being evaluated, but are the global parameters computed during the **build** of the model.

* To perform an input sweep (see :std:ref:`Performing an input sweep`, this operation requires *matplotlib* library installed), run, for example:

.. code-block:: python

  evaluator.perform_input_sweep(x=x_test[49, :], input_id=0, input_interval=[2143, 6956.], output_id=0)

.. figure:: ./images/MetamaterialAntennaPythonInputSweep.png
  :width: 800
  :alt: MetamaterialAntennaPythonInputSweep
  :align: center

  Python API operations: Performing an input sweep: test case - MetamaterialAntennas
  
* To save the model in the native NeurEco binary format:

.. code-block:: python

  save_state = evaluator.save("MetamaterialAntennas/NewDir/SameModel")

* To export the model, run one of the following commands (*embed* license is required):

.. code-block:: python

  evaluator.export_c("./MetamaterialAntennas/MetamaterialAntennas.h", precision="float")
  evaluator.export_onnx("./MetamaterialAntennas/MetamaterialAntennas.onnx", precision="float")
  evaluator.export_fmu("./MetamaterialAntennas/MetamaterialAntennas.fmu")
  evaluator.export_vba("./MetamaterialAntennas/MetamaterialAntennas.bas")

.. warning::
  Once the NeurEco object is no longer needed, free the memory by deleting the object by calling the **delete** method. For the example above, three objects must be deleted:

  .. code-block:: python

    builder.delete()
    evaluator.delete()
    model.delete()